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ABSTRACT 



Chrominance Tcsampling for color images to produce an 
upsan^led chrominance signal from a subsampled chromi- 
nance signal uses luminance to detect edges in the color 
image. The detection of edges produces an Index signal 
which is used to select a particular one of a plurality of 
lowpass filters far filtering a dirominancc signal expanded 
from the subsampled chrominance signal. For interlaced 
video signals a fidd/firame decision is made based upon 
relative motion between the fields of each frame, which 
decision is input to the filter selection process. The resulting 
chrominance samples in the upsampled chrominance signal 
are offset to co-site the saxt^les with the concsponding 
luminance samples at the ou^ut 
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CHROMINANCE RESAMPLING FOR resolution luminance signal is input to a filter selector circuit 

COLOR IMAGES which outputs an Index signal determined by the presence/ 

absence of edges in the picture image. The Index signal is 
BACKGROUKD OF THE DISCLOSURE used to sekct a particular filter from a bank of filters. A 

5 subsanqiled chrominance signal is input to an expander, and 
The present invention relates to video signal fonnat resulting expanded chrominance signal is then input to 

conversion, and more particularly to the upsampling of selected by the Index signaL The output from the 

diroiBinancc components (rf a color image in a color sub- selected filter is the upsampled chrominance signaL For 
sampled domain to a required resolution. interlaced video signals the luminance signal, or optionaUy 

Based on classical three-color theory, color images are jo the chrominance signals, are input to a field/frame decision 
tyjacally represented by three components: Red, Green and circuit to determine whether there is sufficient local motion 
Blue (RGB). Fot most natural images there is a large to treat the two fields of a frame separately or as a progres- 
statistical correlation between the three components. In the frame. Based upon the field^came decision and the 

television world to provide bacJcwaid compatihility with luminance signal die filter selects generates the Index 
monodirome equipment the RGB signals arc lineady trans- 15 signal, which together with the field/frame decision is used 
formed into a luminance (Y) and two color difference (R- Y, to select the particular filter from the filler bank. The output 
B-Y) signals. In almost all the television standards Umai- of the filter is then subjected to a phase shift to co-site 
nance is defined as: chrominance samples with corresponding luminance 

samples to produce the upsampled chrominance signal, 
y=o.299/?40.587£>+o.ii4a ^0 The objccts, advantages and other novel features of the 

The NTSC standard uses I and Q color differences for the present invention are apparent from the following detailed 
color representation: description when read in conjunction with the appended 



/^.27(B-y>*0.74(R-y) 



claims and attached drawing. 
23 BRIEF DESCRIPnON OF THE DRAWING 



FIG. 1 is a blodc diagram view of a system for cfaronu- 
"Pie PAL and TTU-R BT-601 standards use U and V, also nance resampling according to the present invention, 
called C^and C;,, color differences: FIG. 2 is a pictorial view of the position of hmiinance and 

30 chrominance samples in a scan line for (a) 4 :2:2 data and for 
U=o.493(B-Y) (b) 4:4:4 data. 

FIG. 3 is a jpictcrial view of the spatial position of 
v=o.877(R-Y), luminance and dirominance samples for interlaced 4:2:0 

- , . data. 

To reduce the bandwidth requirements for transimtting „ i^ .t. , j . 1 

color images and to reduce the storage requirements, the ^ ^ P}^^ v«.w <)f the vertical and temporal 

color difference signals are horizontally lowpass filtered. In P^^^^f^^f lummance and chrommanoc samples for inter- 



digital images the filtered color difference signals are sub- 



laced 4:2K) data. 

sampled by a factor of two horizontally. The color image HG. 5 is a pictorial view of the vertical and temporal 
representation in this subsanqilcd form is referred to as a ^ position of luminance and chrominance samples after 
4:2:2 signal, and the original unsubsampled signal is chrominance interpolation for interlaced 4:2:0 data accord- 
referred to as a 4:4:4 sign^ In a majodty of video com- ing to the present invention. 

pression standards, such as JPEG and MPEG, the digitized FIG. 6 is a pictorial view of the vertical and ten^ral 

color difference components are lowpass filtered and fol- position of luminance and chrominance samples after 

lowed by subsampling by a factor of two both horizontally chrominance phase shifting according to the present inven- 

and vertically. The color image representation in this sub- tion. 

san^led form is referred to as a 4:2:0 signaL pjQ^ 7 is a block diagram view of chrominance resampler 

The usual procedure for upsampling a subsainpled interlaced video according to the present invention, 

chrominance difference signal is to have an interpolator, or ^ ^ j^^^^ ^^^^ 

data exi^der, that Inserts zero between eve^ ^^^^ field/frame dedsion m^tog according to the 

nance mput samples. A lowpass filter then removes or present invention. 

suppresses the spectralreplicas formed due to the expansion. « , , ^. , x ^ i^x^ 

TOVproccdure does not use the edge information present in ^I?- ^ is ^ block dUgram view of a field/frame decision 

the luminance signal, and performs a bHnd interpolation of Circuit according to the present invention, 

the chrominance which results in soft interpolated color FIG. 10 is a block diagram of a subset of the field/frame 

edges. Such soft interpolated chrominance may be perceived decision circuit of FIG. 9. 

as soft luminance in some color specifications, such as 

EBU/PAL. DESOOPnON OF THE PREFERRED 

Thus what is desired is a chrominance resaiiq>llng of color EMBODIMENT 
images that uses the edge InformaUon present in luminance ^ Referring now to FIG. 1 a chrominance resampling sys- 

to generate sharper color edges in upsanq)led chrominance ^ ^^^^^ y^^g information contained in a 

signals. luminance signal Y in order to generate crisp chrominance 

QinuTMARY OP THR P^FNTION ^^^^ ^ corre^onding upsan4)led dirominance signals 

SUMMARY OF THE INVENTION ^ resolution luminance signal Y is input 

Accordingly the present invention provides chrominance 65 to a filter selector 12. The output from the filter selector 12 

resan^ling for color images that upsamplcs chrominance is an Index signal to a bank of lowpass filters 14 for selecting 

con^onents in a color domain to a required resolution. A full one of the filters for processing the chrominance signaL The 
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selected filter may be either linear or nonlinear. The sub- m,=(^6)*(i'i+Y2+y3) 
sampled chrominance signals Cb^, Cr^ are input to a m2=<^)*(j'i2+i^+w^) 
standard expander 16 where zeros are inserted between m,=(V4)*(n'+yi'+i3') 
every two input samples. The interpolated, or expanded, ^ 
chrominance signal is input to die selected filter from the 5 o^(ma-m, ) 

lowpass filter bank 14 according to the Index signal to The procedure described above may repeatedly be used 
produce the upsampled chrominance signal. This scheme is horizontally two times to convert 4:1:1 material to 4:4:4 
intended for video^picture format conversions, such as 4:2:2 materiaL It also may be qiplied vertically to convert pro- 
to 4:4:4, 4:1:1 to 4:4:4, 4:2:0 to 4:2:2, etc. This process gressive 4:2:0 material to 4:2:2 materiaL 
results in a crisp interpolated color picture without the color lo For vertical intecpolation from 4:2:0 to 4:2:2 where the 
bleeding artifacts seen with the prior art procedure. The source is interlaced video rather than progressive video, the 
Index signal from the filter selector 12 may be (1) a procedure may be as follows. The spatial positions of 
continuously settablc index using an infinitely large number luminance and chrominance samples of an intexlaced 4:2:0 
of lowpass filters, or (2) a discrete index resulting in a finite video frames arc shown in FIG. 3. The luminance and 
number of lowpass filters. is chrominance samples having an "odd" first index bdong to 

An example for a conversion from 4:2:2 to 4:4:4 in a the odd Add of the input video, and the saiiq>les having an 
simple implementation using three linear filters is described "even" first index belong to the even field. The vertical and 
below. The locations of the luminance and chronninance temporal displacement of these sanq>les is shown in FIG. 4. 
san^les in a scan line for the two formats are shown in FIG. The chrominance interpolation for interlaced 4:2:0 to 
2. The task of the chrominance resampler is to derive the 20 4:2:2 may be conceptuaUy thought of as two steps. These 
chrominance samples that ate not present in the 4:2:2 data, stqjs may casUy be ma^cd into a single step of intapola- 
but required in die 4:4:4 representation, i.e,, the even values tio^- ^ fhc first step the chrominance samples are intcrpo- 
of the chrominance san^lcs. lated vertically to double the ii^ut sample rate and associate 

The missing chrominance samples may be generated to ^ J^eCb and one a sample xvitheadiY^^ 
meetcertaingoals-Ifthegoalistomakethelocalgradients ^ Cb, Cr samp^ are not oo-sited wiA 
of chrominai^e the same as that of luminance: «^ showninFIG. 5. In the second stq? these Cb, Cx samples 

are given phase shifts so that the resultant Cb, Cr sainples are 
Cb2 - cbi Cf2 - Cri Yi-Yi co-sitcd With the luminance samples, as shown in FIG. 6, to 

dh2-c^ ^Cr2-Cr3 ^ 'n-Y^ pToducc the 4:2:2 output 

30 The first step of the interpolation depends upon the picture 
In this case samples Cb2 and a2 may be generated through characteristic m die neighborfiood of the intapolated 
the procedure* chrominance sanq^le. The lummance samples of the mtcr- 

' laced video, and optionally the chrominance samples, are 

if (Yi-Y3>T*CYi+Y3> then used to decide if there is motion between the two fields of 

35 a frame. This decision is made on small blocks of die frame 
^ A (Y2-Y3ycyi-Y3) in a field/frame decision circuit 20, as shown in FIG. 7. If 

there is not enough motion between fields, the block may be 
^ considered progressive for the purposes of chrominance 

inteipolation. In which case the vertical interpolation of 
40 chrominance uses the chrominance samples from both 
endif fields. However if there is significant motion between fields, 

the block is considered interlaced and the vertical interpo- 
Cfc2=^*C6i-Ki-<)*c&3 lation of chrominance uses chrominance samples from the 

Cra^*(CriMi-<)*M ssooQ field as the interpolated chrominance sample. The 

45 field/frame decision from the decision circuit 20 is input as 
T is a contrast threshold, 0<r<l, and may be set to a value an additional input to the filter selector 12 and the lowpass 
of 0.1, and C is a variable that controls the filter coefficients. filter array 14. 

The first stqp above determines the presence of a luminance The field/frame decision is made on small blocks of pixels 
edge, and the variable C is set for linear interpolation when as shown in FIG. 8. These blocks of sanq)les are input to the 
no edge is present field^rame decision circuit 20, shown in greater detail in 

The position of an edge in the color image may also be FIG. 9. The upper four luminance pixels in the block are 
estimated using the luminance signal as indicated below, and input to an upper block decision circuit 22 while the lower 
dien the edge position information may be used to perform four luminance pixels in the block are input to a lower block 
the chrominance signal interpolation. For exanq)le the ' decision circuit 24. The outputs from the upper and lower 
wci^ting factor C in the procedure described above may be 55 block decision circuits 22, 24 arc input to respective OR 
found from the edge location estimation to be: gates 30, 32. The respective chrominanoe signals are input 

to respective complete block decision circuits 26, 28, with 

;=o.75{i+*%^uC4+e')}, outputs provided to both of the OR gates 30, 32. The 

field/frame decision of *Upper** block depends on the cat- 
where s is given by: egOTization of luminance samples from "Upper" block as 

s A (m3+2m,^-3m,m2ya' ^ ficld/framc decision on chrominance samples 

* * from "Complete" block Likewise the field/frame decision 

and m^, m^ and m3 are the first, second and third moments of '"Lower"' block depends on the categorization of lumi- 
of the luminance signal respectively, and 0 is the standard nance signals from **Lower^ block as well as the field/frame 
deviation of the luminance signal. For simplicity the three 65 decision on chrominance samples from "Complete" block, 
luminance samples may be used to compute the moments Arepresentative field/frame decision drcoit 22, 24, 26,28 
and Sigma as follows: is shown in greater detail in FIG. 10. The first and second 
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inputs are input to a first summing circuit 34, the third and 
fourth inputs are input to a second summing circuit 36, the 
first and third inputs are input to a third summing circuit 3S, 
and the second and fouith in^nits are input to a fourth 
summing circuit 40. The outputs from the first and second 
summing circuits are input to a first contrast circuit 42, while 
the outputs from the third and fourth summing circuits are 
input to a second contrast circuit 44. Contrast may be 
defined as: 

Coiitrast(i,y)A(lx-yiy(x+y) 

The outputs from the two contrast circuits 42, 44 arc input 
to a conoparison circuit 46 that determines the fidd/firame 
decision based on four inputs by making two ^larger tiian" 
con^arisons as follow: 

'*field» if(lfl + c-fc-*fl>lo + i»-c-<fl),and 
Dcci5iaa= (lfl+c-*-d> rHfl + 6 + c + rfD 

**&anie" otherwise 

where T is the contrast threshold. 

The interpolation is performed based on the field/firame 
decision. In the interpolation process the chrominance 
samples that exist in 4:2:0 are retained in the upsan^led 
domain. The generation of chrominance samples at sites 
(2,1), (3>1), (6,1), etc. depends on the flame^field decision 
made in the first step. If a firame decision was made^ the 
inteipolatLon uses the spatially neighboring samples from 
lx)th fi.elds for interpolation. For example Cb(3,l) is inter- 
polated from Cb at sites (1,1), (4,1) and (5,1). If a field 
decision is made, the interpolation uses the spatially neigh- 
boring samples from the same field, i.e., for Ct>(3,l) the 
interpolation is perfonned using (1,1) and (5,1). A large 
variety of linear and nonlinear filters may be used for this 
interpolation. 

Going from the interpolated chrominance samples shown 
in FIG. 5 to those shown in FIG. 6 requires vertically phase 
offsetting the chrominance samples. Tills may be performed 
in a number of standard ways, such as (1) linear interpola- 
tion using two samples or (2) longer fi.mte inq)ulse response 
filters. The phase offsetting step also uses the frame/field 
decision made earHer. If the decision is *^eld*', the 
wdghted-sum confuted for the phase shifting uses samples 
from the same field. Otherwise if the decision is **frame", the 
wdghted-sum computed for the phase shifting uses samples 
from both fidds. 

Thus the present invention provides a chrominance resa- 
mpler for color images that provides crisp chrominance 
edges by sdecting a filter for lowpass filtering that is a 
function of edges of the color image using information from 
the limiinance component of the color image. 

What is claimed is: 

1. A chrominance resan:q)ler of the type that interpolates 
a subsampled chrominance component of a color image to 
pXMluce an upsampled chrominance component comprises: 

a plurality of lowpass filters coupled in paralld to recdve 
the interpolated subsampled dirominance component 
as an input and to provide the upsampled chrondnance 
component at an output from one of the lowpass filters 
in response to a selector signal, each lowpass filtesr 
having a different characteristic; and 

means for generating the sdector signal from a luminance 
component of the color image, the generating means 
detecting the position of edges in the color image from 
the luminance component and providing the selector 
signal as a function of the position of the edges. 
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2. The dirominance resan^ler as redted in daim 1 fiirther 
coTaprisiRg means for detecting motion between fields of the 
color image when the color image is in interlaced f(Biaat to 
generate a field/frame signal, the field/frame signal being 

s input to the lowpass filters as part of the sdector signal and 
to the generating means as a control signal to determine 
whether chrominance samples from just one fidd or from 
the entire fr^me of the color image are used for interpolation. 

3. The chrominance resampler as redted in daim 2 further 
coursing means for phase shifting the upshifted chromi- 
nance conq>onent to be co-sited with the corresponding 

l uminan ce COn^OUent 

4. The chrominance resan^ler as redted in claim 2 
wherein the detecting means con^irises: 

means for combining inter fidd samples of the luminance 
component to produce an interfidd ou^ut and for 
combining intraiidd saiiq>les of the luminance conq)0- 
nent to produce an intxafidd output; and 

means for comparing the interfreld output with the 
intrafidd output to generate the field/frame signal, the 
^ field/frame signal indicating **fidd'* when motion is 
detected and indicating ''frame" otherwise. 

5. The chrominance resampler as redted in claim 4 
wherein the combining means comfBises: 

first and second summers for combining adjacent vertical 
^ luminance samples from adjacent frelds of the color 
image to produce the interfield ou^ut; 
second and third summers fc^ combining adjacent vertical 
luminance san^les finom the same frdd of the color 
^ image to produce the intrafield ou^t 

6. Tlie chrominance resan^ler as redted in claim 5 
wherein the comparing means comprises a comparator that 
provides the **fieijd" indicator for the fi.dd^rame signal when 
the interfidd output is larger than the intrafrdd ou^ut and 
the interfidd output is greater than tiie sum of the vertical 
pairs from both fields of the color image times a threshold 
value, and otherwise provides the '"frame" indicator for the 
field/frame signal 

7. A method of chrominance resampling that indudes the 
^ step of interpolating a subsampled chromioanoe con^nent 

of a color image to produce an upsampled chrominance 
component comprising the steps of: 

generating a sdector signal from a luminance componeat 
of the cdor image as a function of detected edges of the 
color image; and 
sdecting a filter from among a plurality of lowpass filters 
coupled in parallel to recdve the interpolated sub- 
sanded chrominance con^onent as a function of the 
sdector signal, each lowpass filter having a different 
SQ characteristic; and 

providing the upsan^kd chrominance component at an 
ou^ut of the sdccted filter. 

8. The method as redted in daim 7 further conqirising the 
steps of: 

33 detecting motion between fidds of the coIot image when 
the color image is in mterlaced fomat to generate a 
fiddyframe signal; 
inputting the fidd^frame signal to iht plurality of lowpass 
filters as part of the sdector signal and to the sdecting 

60 step as a control signal to detennine whether chromi- 
nance samples from just one Add or the entire firame of 
the color image are used in the interpolating step. 

9. The method as recited in daim 8 further comprising the 
step of phase shifting the upshifted chrominance con^nent 

65 to be co-sited with the ooiresponding luminance cotmponent 

10. The method as recited in daim 8 wherdn the detecting 
step con[iprises the steps of: 



04/07/2004, EAST Version: 1.4.1 



5,7] 

7 

combining interfield samples of the luminance component 
to produce an inteifidd output and intrafidd samples of 
the luminance component to produce an intrafield out- 
put; and 

comparing the interfield and intrafield ou^uts to generate 
the ficld/firame signal, the field/frame signal indicating 
**fidd" when motion is detected and indicating •*frame" 
otherwise. 

11. The method as recited in daim 10 wherein the 
combining step con^ses the steps of: 
summing adjacent vertical luminance samples from the 

fields of the color image to produce the interfield 

output; and 



;,687 

8 

summing adjacent vertical luminance samples from the 
same field of the cdor inoage to produce the intrafield 
output 

12. The method as recited in claim 11 wherein the 
comparing step conquises the steps of: 

providing the **field" indicator for the field/frame signal 
when the interfield output is larger than the intrafidd 
ou^ut and the sum of the vertical lummance samples 
from both fields of the color image times a threshold 
value; and 

^ providing the '*frame" indicator for the fldd/frame signal 
otherwise. 

***** 
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